Search for: All records

An interesting behavior in large language models (LLMs) is prompt sensitivity. When provided with different but semantically equivalent versions of the same prompt, models may produce very different distributions of answers. This suggests that the uncertainty reflected in a model's output distribution for one prompt may not reflect the model's uncertainty about the meaning of the prompt. We model prompt sensitivity as a type of generalization error, and show that sampling across the semantic concept space with paraphrasing perturbations improves uncertainty calibration without compromising accuracy. Additionally, we introduce a new metric for uncertainty decomposition in black-box LLMs that improves upon entropy-based decomposition by modeling semantic continuities in natural language generation. We show that this decomposition metric can be used to quantify how much LLM uncertainty is attributed to prompt sensitivity. Our work introduces a new way to improve uncertainty calibration in prompt-sensitive language models, and provides evidence that some LLMs fail to exhibit consistent general reasoning about the meanings of their inputs. 

Abstract All-solid-state sodium batteries (ASSSBs) are promising candidates for grid-scale energy storage. However, there are no commercialized ASSSBs yet, in part due to the lack of a low-cost, simple-to-fabricate solid electrolyte (SE) with electrochemical stability towards Na metal. In this work, we report a family of oxysulfide glass SEs (Na 3 PS 4− x O x , where 0 <  x  ≤ 0.60) that not only exhibit the highest critical current density among all Na-ion conducting sulfide-based SEs, but also enable high-performance ambient-temperature sodium-sulfur batteries. By forming bridging oxygen units, the Na 3 PS 4− x O x SEs undergo pressure-induced sintering at room temperature, resulting in a fully homogeneous glass structure with robust mechanical properties. Furthermore, the self-passivating solid electrolyte interphase at the Na|SE interface is critical for interface stabilization and reversible Na plating and stripping. The new structural and compositional design strategies presented here provide a new paradigm in the development of safe, low-cost, energy-dense, and long-lifetime ASSSBs.